KOLMOGOROV COMPLEXITY AND THE GARDEN OF EDEN 

THEOREM 



ANDREY ALPEEV 

Abstract. Suppose t is a cellular automaton over an amenable group and a 
finite alphabet. Celebrated Garden of Eden theorem states, that pre-injectivity 
of r is equivalent to non-existence of Garden of Eden configuration. In this pa- 
per we will prove, that imposing some mild restrictions , we could add another 
equivalent assertion: non-existence of Garden of Eden configuration is equiv- 
alent to preservation of asymptotic Kolmogorov complexity under the action 
of cellular automaton. It yields a characterisation of the cellular automata, 
which preserve the asymptotic Kolmogorov complexity. 
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1. Introduction 

Suppose G is an amenable group and A is a finite set, called an alphabet. Suppose 
we fixed some F0lner sequence {F n }. For an infinite element x S A G in the discourse 
of the Kolmogorov complexity it is very natural to consider some kind of mean 
information along the F0lner sequence: 

i / \ ,. C(x\ F J 
hc(c) = lim sup — — — — 

n— >oo \-^n\ 

In this way we should make some assertions, to guarantee well behaviour of the 
defined quantity. If we are working with a group like Z d we do not care about com- 
putability. Passing to more general case we have to explicitly require our group to 
be computable. Then, there is a choice of an object to measure the complexity: the 
string, obtaining by listing of letters, marking elements of given F n in some order, 
or the whole object: the map from F n to A. In the second case we get an additional 
information from the configuration of F n . To avoid such troubles, we introduce the 
notion of the modest F0lner sequence, that is such F0lner sequence, that complex- 
ities of its elements are asymptotically negligible to their sizes: C(F n ) = o(\F n \). 
This will help us establish equivalence of the two definitions of the asymptotic 
Kolmogorov complexity. 

It is not hard to see, that asymptotic Kolmogorov complexity do not increases 
under the action of cellular automatonf proposition ITUf . The point of interest is to 
describe those cellular automata, which preserves asymptotic complexity in terms 
of some well-known properties of cellular automata. The first result of such kind, 
known to author, contained in the paper [2], there proved, that every invertible cel- 
lular automaton over Z with finite alphabet preserves the asymptotic Kolmogorov 
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complexity, which is defined in the following way: 

r C{xi) 
am sup — — - 

i — ^oo 22 ~t~ 1 

there C{xi) is the Kolmogorov complexity of the restriction of x to the segment 
[-hi]- 

The Garden of Eden theorem states, that for the cellular automaton with finite 
alphabet over an amenable group surjectivity(that is non-existence of Garden of 
Eden configurations) and pre-injectivity are equivalent (cellular automata is called 
pre-injective, if every two different configurations with same images under action 
of cellular automaton, differ on infinite set). We will prove in this paper, that the 
class of cellular automata, which preserve asymptotic Kolmogorov complexity is 
exactly the class of surjective and pre-injective cellular automata. 

This paper organized in the following way. In section [2] we discuss the notion of 
a computable group and prove existence of a computable F0lner sequences in com- 
putable groups. In section [3] we remind the notion of the Kolmogorov complexity 
and its basic properties. Also, there is proved a crucial proposition [51 which equip 
us with the lower bounds of the preimages of a computable functions. In section 0] 
we remind some definitions, concerning cellular automata. In section [5] we discuss 
properties of entropy and asymptotic Kolmogorov complexity. At first, we give the 
definition of the modest F0lner sequence, Then, we remind the definition of the 
entropy and define the asymptotic Kolmogorov complexity. After, we prove the 
basic properties of the asymptotic complexity. In proposition [TU] we prove, that the 
entropy of effectively closed set bounds the asymptotic complexity of its elements. 
After it we remind the Garden of Eden theorem for amenable groups and Curtis- 
Hedlund-Lyndon theorem. In the end of this section we give the proof of the main 
result, theorem [3j which states, that asymptotic Kolmogorov complexity along the 
modest F 0lner sequence is presserved by the action of the cellular automaton over 
the computable amenable group if and only if this automaton is preinjective. 

2. Computable amenable groups 

Definition 1. A pair of a countable or finite set and a bijection of it with an 
enumerable subset of the natural numbers called a constructible set. 

Definition 2. Let G be some some group structure on N, such that is the identity 
element. Then, if function (x, y) i-y x o y is computable, we will call the group G 
computable. 

It is obvious, that the operation x M- x^ 1 in computable group is computable 
function. 

Throughout this paper we will assume, that G is computable, if not, we could 
enhance our computability class by a suitable oracle. 

Definition 3. Suppose G is a countable group. We call a sequence {F n } of its 
finite subsets a F0lner sequence for this group, if for every g G G 

hm »^Lo 
n->oo \F n \ 

there A denotes the symmetric difference. 

Definition 4. We call a countable group amenable, if it has a F0lner sequence. 
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Proposition 1. Suppose, G is a countable amenable group, {F n } — its F0lner 
sequence. Then for every finite subset S of G we have 

lim Ig^fiUo 

n^oo \F n \ 

Proof. Since S = {si, . . . , Sk}, we have 
F n SAF n = |J (F„ Si \ F„) U p| (F n \ F nSj ) C |J {F n s t AF n ) U (F„ \ F nSl ) c 

1=1 j = l 2 = 1 

fe 



c |J (F nSi AF n ) U (F n AF nSl ) 



From which we could infer, using the definition of F0lner sequence 

.. \F n SAF n \ ^ J2i-i\ F n^F n \ + \F n AF nSl \ 
km ; : < km ; ; = 

F n -«->«, \F, 



□ 



Proposition 2. Suppose G is an infinite countable amenable group, and {F n } is 
a F0lner sequence on it, then \F n \ — > oo 

Proof. Suppose, on the contrary, for some constant c we have infinitely many indices 
i, such that |i^| < c. Then, passing to subsequence, we will get a F0lner sequence 
{F ni } with \F nk \ < c. Take some finite subset S of G with \S\ > 2c we get a 
contradiction with the proposition [T] 

\F nz SAF nt \ 

\F ni \ ~ 

for every i. □ 

Every finite subset of the natural numbers could be computably encoded by the 
natural numbers, so, they forms the constructible set. It is clear, that set-theoretic 
operations with the finite subsets are computable, and function (A, B) h-> AB there 
A and B are the finite subsets of computable group is computable too. 

Proposition 3. There exists a computable F0lner sequence , with \F n \ > n in every 
computable amenable group. 

Proof. Fix some computable enumeration of the finite subsets. Let F n be the first 
set, such that 

\F n iAF n \ 1 . 
— n=n — S — i° r every i, < i < n 
\F„\ n 

and |F„| > n. 

F n is a desired computable F0lner sequence. □ 

3. KOLMOGOROV COMPLEXITY 

We will use the notation C{x) for the plain complexity and C(x\y) for the con- 
ditional complexity, for the definition see [3 , definition 2.1.2, p. 106 . Plain com- 
plexity of the pair (x, y) is denoted as C{x,y), for the definition see [3], example 
2.1.5, p. 109. 

Proposition 4. For the plain Kolmogorov complexity the following statements hold: 
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(1) C{x) < log 1 + 0(1) 

(2) if f is computable, then C(f(x)) < C(x) + 0(1) 

(3) There exist a constant c, such that \{x\C(x) < n}\ < c2 n 

(4) C(x,y) = C(y,x) + 0(l) 

(5) C(x,y) <C(x)+C(y)+2\ogmm(C(x),C(y))+0(l) 

(6) C(x\y)<C(x)+0(l) 

(7) C(x, y) = C(x) + C(y\x) + 0(max(log C(y), log C(x))) 

Proof. For the proofs, see [3]: 

(1) is theorem 2.1.2 on the page 108, 

(2) and (3) are simple consequences of the definition of plain complexity, 

(4) is follows from (2) and definition of complexity of pair, 

(5) is proved in example 2.1.5 on the page 109, 

(6) is from exercise 2.1.5 on page 113, 

(7) is a slight reformulation of the theorem 2.8.2 on the page 190. 

□ 

Proposition 5. Suppose, f is a computable function. If x — f(y), then 

logirV)! > C(y) - C{x) - 0(\og\C(y) - C(x)\) 

Proof. Fix some computable enumeration t(x,i) (i < 0) of the preimage of the 
function /, such that if t(x, i) does not fail, then t(x,j) does not fail for all < j < i 
and t(x,i) + 1 t(x,j) (if both do not fail) for i ^ j . Then, there exists i < |/ _1 (x)|, 
such that y — t(x, i). Using proposition 01 we get 

C(y) = C(t(x, »)) < C(x, i) + 0(1) < C(x) + logl/" 1 ^)! + loglogl/" 1 ^) + 0(1), 
so 

logir 1 ^)! + loglog|r x (x)| > C(y) - C(x) + 0(1) 
which implies desired estimate. 

□ 

Proposition 6. Suppose, that {Vi} i€l is an enumerable family of enumerable sets, 
there I is some constructible set. Suppose, there exists a function f : N — > N, such 
that for every i G / we have \Vi\ < f(i), then for every x £ V{ the following holds: 

C(i,x) < \og\Vi\ + C{i) + 0(logC(i)) 

Proof. Follows from proposition [4] □ 

4. Cellular automata 

Suppose G is a group and A is a finite set. We will call A an alphabet. We will 
call A G a configuration space. There is a natural left action of G on A G defined in 
the following way: 

{gx){g') = x{g- 1 g'). 

Definition 5. We will call a word every map from any finite subset of G to A. 

The set of all words is denoted by A* . For a word w we denote dom w its domain. 
By \w\ we will denote the size of its domain. Consider a word w, by the definition 
it is a map from the finite set {gi, ■ ■ ■ g\ w \} to the alphabet A 
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We will endow A G with the product topology, assuming that A endowed with the 
discrete topology. By Tychonoff theorem, A G is a compact Hausdorff topological 
set. Sets 

U(w) — {x £ A G \ x\domw = w} 

are clopen and forms the base of the topology on A G . 

Assuming G is computable, it is obvious, that every word could be computably 
encoded by some natural number. 

Since all finite subsets of natural numbers and all words forms constructible sets, 
it make sense to consider the Kolmogorov complexity of such objects, and we will 
use the same notation as fo plain complexity: C(B), C(w), there B is a finite 
subset, w is a word. 

Consider a word w, by the definition it is a map from the finite set {<?i, . . . gy. 
Assume, that <?; < gj for f < i < j < \w\ (remind, that group G is defined on 
the set N). Then, string w{g\)w{g2) ■ ■ .w{g\ w \) is called a content of word w, and 
denoted as contw. By the definition, for word w holds C(w) — C (dom w , cord w) . 

Definition 6. Cellular automaton over group G and the finite alphabet A is a map 
t : A G — » A G such that there exist a finite set S C G, and a map \i : A s — » A, for 
that the following equality holds: 

T {x){g) = ti{g~ 1 x\ s ) 

Set S is called a memory set, ji is called a rule. 

Definition 7. Suppose t is a cellular automaton with a memory set S and a rule 
fj,, Suppose, w is a word, such that dom w = BS for some subset B of G. Then we 
will define 

T (w){g) = LL{g' 1 x\ s ) 

for g G B. So it is no more, than the restriction of the action of the cellular 
automaton to the word w. 

Evidently, if G is a computable group and r is a cellular automaton over it, then 
restriction of r on words is computable function. 



5. Entropy and asymptotic Kolmogorov complexity 
We want our F0lner sequence not to affect Kolmogorov complexity 
Definition 8. We will call a F0lner sequence {F n } a modest Foincr sequence, if 

hm = 

It obviously follows from the proposition [31 that there exist at least one modest 
F0mer sequence. 

It would be convinient for us to use the following notation for the restriction 
of configuration x G A G to the subset B of G: x\b- If B is a finite set, then this 
restriction is a word, so, if in addition, G is a computable group, then this would 
justify the notation C(x\b)- We would use the same notation for the restriction of 
words as well: w\b if B is a subset of the domain of word B. 

Let us fix some modest F0lner sequence {F n }. 
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Definition 9. Consider some subset X of A G . We will define its entropy as 

h(X) = lim S up l ° g }X\ Fj 

n— )-oo \ n\ 

Proposition 7. For the entropy the following holds: 

(1) h(X) < \og\A\ 

(2) h(A G ) = log|A| 

(3) h(r(X)) < h(X). 

Proof. For the proof see p], propositions 7.7.2 and 5.7.3 on pages 125,126. □ 

Definition 10. For the element x G A G we will define its asymptotic complexity 
as 

u ( \ v c ( x \fJ 
nc(x) — am sup — — — — 

Proposition 8. For every x G A G we have 

7 / \ C (cont(x\ Fn )) 
nc(x) = limsup j— -j 

n— >oo \^ n\ 

Proof. It is enough to prove, that 

,. C(x\ Fn )-C(cont(x\ Fn )) 
nm j j = 

n^oo \F n \ 

Using proposition [4] we have 

r \C(x\ Fn )-C(cont(x\ Fn ))\ 
urn j : = 

n-^oo \F n \ 

y \C(F n \(x\ F J) + Q(\og(m a x(C(F n ),C(cont(x\ F J))))\ 
= lim j j = 

rwoc \F n \ 

the last equality is true, since 

C(F n \(x\ F J) < C{F n ) + 0(1) = o(|F n |), 
(by the proposition 0] and the definition of modest F0lner sequence), and 
C(cont(x\ F J) < \F n \ \og\A\ + 0(1). 

by the proposition |¥J 

□ 

The following proposition allows us to estimate the difference of the complexities 
of two words in terms of the symmetric difference of their supports. 

Proposition 9. Suppose, w± and u>2 are two words, then 

\C(wi) — C{w2)\ < \dom(wi)Adom(w2)\ • log|A| + 

+ 0(max(C(cont(wi), cont(w2)))) 

Proof. Evidently, there exists an algorithms, which recovers w\ from the triple W2, 
dom(w\) and cont(wi\dom(w 1 \w 2 ))- From the proposition 2] follows 

C{w x ) < C{w 2 ) + 0(C(dom(w 1 ))) + C{w{) + \og\A\ ■ \dom(w 1 )Adom(w 2 )\. 

By symmetry we get 

C(w 2 ) < C{w x ) + 0(C{dom(w2))) +C(w 2 ) +log|A| • \dom(wi)A dom(w 2 )\. 
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Combining this two we get the desired estimate. 



Let us endow A G with the Bernoulli measure v. 

Proposition 10. The following holds: 

(1) for every x £ A G we have hc(x) < log|A 

(2) for almost every x G A G we have hc(x) = log|^4| 

(3) for every x £ A we have hc(r(x)) < hc(x). 

Proof. The following inequality implies the first statement : 
C(cont(x\ F J) < log|F n ||A| +0(1), 
For the proof of the second statement consider a set 

E £ = {je A G \hc(x) < (1 - e) log|A|} 

We have 

E E = {J f]{x £ A^CicontixlFj)) < (1 - e)\Fj\ logL4|} 

But since 

^({a; £ A G |C(cont(x|Fj)) < (1 - e)\Fj\ log|A|}) < 

< |{y|C(y)<(l- £ )|F J -|log|A|}| < e | Fj .| ^ 

We have v{E £ ) — 0. Taking a sequence e„ — > 0, we see, that v(U i>0 E ei ) = 0, 
almost every x £ A G has /ic(a;) = log|^4|, because hc(x) < \ A\ for every x. 
For the proof of the third statement, consider 

u i \ v C ( x \fJ 
hc(x) — hmsup — — — — 

n— >oo | -t*ri | 

and 

hc(T(x)) = hmsup ■ 



\Fn\ 
but 

t(x)\ F „ = r(x\ Fn s), 

so 

C(r(x)\ Fn )<C(x\ FnS ) + 0(l). 
This mean, it is enough to prove, that 

]im C(x\ Fn s)-C(x\ Fn)=o 

n^oo \F n \ 

Proposition [S] implies 

\C(x\ FnS ) - C(x\ Fn )\ < \F n SAF n \ ■ \og\A\ + 0(max(C(F„), C(F n S))) 
By the definition of the F0lner sequence, 

\F n SAF n \ = o(\F n \) 
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Using proposition |4j since F n S is computable from F n , and by the definition of 
modest F0lner sequence we have 

C(F n S)<C(F n ) + 0(l)=o(\F n \), 

which finishes proof. 

□ 

Definition 11. An open set V is called an effectively open set, if 

V= (J U(w) 
wew 

for some enumerable set W of words. 

A closed set is called effectively closed, if it is a complement of an effectively 
open set. 

Proposition 11. If X is an effectively closed set, then for every x 6 X we have 

hc(x) < h(X). 

Proof. It is clear, that \X\ Fn \ < 

Let us fix some rational q > h{X). By the definition of the entropy, there exists 
a constant c, such that 

\X\ Fn \ < c2«l i ""l 

If there exists an enumerable family of enumerable sets {V/}, there / £ {F%, F2, . . .}, 
such that X\f cVf and Vf„ | < c2 9 ' i= ™' for all n, then by the proposition[6]we would 
have for y £ V n 

C(y,F n )<0(C(F n )) + q\F n \ 
and therefore, for every x £ X 

hc(x) < q 

But since q is an arbitrary rational, bigger than h(X), we would have hc(x) < hc(X) 
for every id. 

Let us prove existence of {Vf}. X is an effectively closed set, so 

A G \X= (J !/(«;) 

for some enumerable set of words W. Consider a word v with domv = f. Suppose, 
that U(v) n X = 0. Then, 

U{v) C |J U(w), 

and since it is an open covering of the compact set, we have, U(v) could be covered 
by some finite subset. This mean, that if U(v) Pi X = 0, we could realize it in a 
finite time. Therefore, the set of such v-s is enumerable(and computably depends 
on n). Really, assertion, that U(v) is covered by U(wi), . . . , U(wk) is equivalent to 
the fact, that for every word t with domain 

k 

E = dom(v) dom(u>i) 
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from the fact, that (t\domv) = v follows, that there exists 1 < i < k, such that 
(t\domwi) — Wi This means, that sets 

k 

V f , k = (A G \\JU(w i ))\ f 

i=l 

forms the computable family of the finite sets, and we could get Vf to be the first 
V f , h with \Vf, k \ <c- 2^1/1. 

□ 

Definition 12. A cellular automaton r over a group G and an alphabet A is called 
pre-injective, if for every x, y £ A G , such that x and y coincides outside some finite 
set, and t(x) — r(y), we have x = y. 

Definition 13. A configuration x 6 A G is called a Garden of Eden configuration, 
if its pre-image is empty. 

Theorem 1 (The Garden of Eden theorem). For a cellular automaton t over a 
group G and an alphabet A the following are equivalent: 

(1) There is no Garden of Eden configuration. 

(2) t is pre-injective. 

(3) h(r(A G )) = h(A G ). 

Proof. See [T] theorem 5.8.1 on page 128. □ 

Theorem 2 (Curtis-Hedlund- Lyndon theorem). Suppose t is a map from A G to 
itself there G is a group and A is a finite set. Then the following are equivalent: 

(1) t is a cellular automaton. 

(2) r is continuous and shift-invariant, that is for every x £ A G and g 6 G we 
have r{gx) = gr{x). 

Proof. See [T], theorem 1.8.1 on page 20. □ 

Proposition 12. Suppose, that cellular automaton t has a Garden of Eden con- 
figurations. Then for some x £ A G we have hc(r(x)) < hc(x). 

Proof. By the Garden of Eden theorem, if there exists a Garden of Eden config- 
uration, then h(r{A G )) < h(A G ). By the proposition HU1 there exist an element 
x G A G with hc{x) = \og\A\. The set t(A g ) is a closed subset of the set A G , 
because it is image of the compact set under the action of the continuous map. 
Since the statement U(v) n t(A g ) = is equivalent to r _1 (u) = 0, we have, 
that the set t(A g ) is effectively closed. So, by the proposition [TT] we have, that 
hc{r{x)) < h(r(A G )) < log\A\ = hc{x). □ 

Proposition 13. Suppose we have x £ A G , such that hc(r(x)) < hc{x). Then the 
cellular automaton is not pre-injective. 

Proof. Without loss of generality we may assume, that S — S^ 1 , and that S con- 
tains the identity. Since hc(r(x)) < hc{x), there is a subsequence {n^} and two 
constants a < b, such that 

C(r(x)\ Fnk ) ^ cf) ^ C{x\ Fnk ) 

\Fn k | l-^rifc I 
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Using the proposition[9]we could assume(maybe, passing to subsequence and slightly 
modifying constants a and b), that 

C(t(x)|,.J , C(x\r. t s) 

-lKj^ <a<b< -iF^- 

then, since 

T ( x \Fn k s) = T ( x )\F nk 
we could use the proposition [5] and denoting y — t{x) we get 

logl-r-^ykjl > C{x\ FnS ) - C(y\ Fn ) - 0(log\C(x\ FnS ) - C(y\ F J\) 

which imply, that for some big enough N and positive constant c, for every k > N 
the following holds: 

logl-r-^ykjl >c-|F„ fe | 

so 

|r- 1 (ykj|>2 c ^l 

Consider a set T of pairs of words (v,w), there f(v) = w, dom(w) = F nk SS and 
v \f„ k — T ( x )\F„ k ■ Its cardinality is at least 2 C ' F ™<=' for k > N. That is, for big 
enough k, by the pigeonhole principle there exists at least two pairs {v\,W2) and 
(v2,W2) in T, such that v\ and i>2 coincides outside the the set F nkl and w± = W2- 
Really, w\ and W2 coincides on F nk , and we have |A|°^™ fe '^ variants for the filling 
F nk SS \ F nk and F nk S \ F nk by elements of A. 

Extending w\ to some x\ € A G and 102 to some X2 G A G in such a way, that 
they will coincide outside F nk , we will get, that automaton is not pre-injective, 
since t(xi) = t{x2)- 

□ 

Combining last two propositions and the Garden of Eden theorem, we get the 
following result 

Theorem 3. Suppose, that G is an infinite computable amenable group, a cellular 
automaton t defined over it, and a modest F0lner sequence selected(that is, such a 
F0lner sequence, that C(F n ) = odi 7 ^!),). Then the following are equivalent 

(1) t has no Garden of Eden configurations 

(2) t is pre-injective 

(3) h(r(A G )) = h(A G ) 

(4) hc(r(x)) = hc{x) for every x G A G . 
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